Built-in AI  |  AI on Chrome  |  Chrome for Developers

🌈 Abstract

The article discusses the benefits and challenges of integrating AI models, including large language models (LLMs), directly into web browsers. It highlights the advantages of built-in AI, such as ease of deployment, access to hardware acceleration, and the ability to perform AI tasks on-device for improved privacy, user experience, and offline usage. The article also covers hybrid approaches that combine on-device and server-side AI, depending on factors like complexity, resiliency, and graceful fallback. It provides an overview of the browser architecture and APIs being developed to support built-in AI, and outlines potential use cases like AI-enhanced content consumption and AI-supported content creation.

🙋 Q&A

[01] Benefits of Built-in AI

1. What are the key benefits of built-in AI for web developers?

• Ease of deployment: The browser distributes and manages the models, taking into account device capabilities and handling updates, so developers don't have to worry about downloading or updating large models.
• Access to hardware acceleration: The browser's AI runtime is optimized to leverage the available hardware (GPU, NPU, or CPU) for the best performance.

2. What are the benefits of running AI on-device?

• Local processing of sensitive data: On-device AI can improve privacy by processing sensitive data without sending it to a server.
• Snappy user experience: On-device AI can provide near-instant results, improving the user experience.
• Greater access to AI: On-device AI can enable users to access premium AI features without additional cost to the developer.
• Offline AI usage: Users can access AI features even without an internet connection.

[02] Hybrid AI Approaches

1. When might developers consider a hybrid approach of on-device and server-side AI?

• Complexity: Specific, approachable use cases are easier to support with on-device AI, while complex use cases may require server-side implementation.
• Resiliency: Use server-side by default, and use on-device when the device is offline or on a spotty connection.
• Graceful fallback: Offer server-side AI for users with older or less powerful devices that cannot run all models optimally.

[03] Browser Architecture and APIs

1. What infrastructure has Chrome created to support built-in AI? Chrome has created infrastructure to access foundation and expert models for on-device execution, powering features like "Help me write" and upcoming on-device AI APIs.

2. What types of APIs will be provided for built-in AI in Chrome?

• Task APIs: Designed to run inference against the best model for a specific assignment, such as translation or summarization.
• Exploratory APIs: Provide access to experiment with the built-in Large Language Model (Gemini Nano) and fine-tune it for specific tasks.

[04] Use Cases for Built-in AI

1. What are some ways built-in AI can benefit web developers and their users?

• AI-enhanced content consumption: Summarization, translation, question answering, categorization, and characterization.
• AI-supported content creation: Writing assistance, proofreading, grammar correction, and rephrasing.